An Algorithm for Finding the Distribution of Maximal Entropy

An algorithm for determining the distribution of maximal entropy subject to constraints is presented. The method provides an alternative to the conventional procedure which requires the numerical solution of a set of implicit nonlinear equations for the Lagrange multipliers. Here they are determined by seeking a minimum of a concave function, a procedure which readily lends itself to computational work. The program also incorporates two preliminary stages. The first verifies that the constraints are linearly independent and the second checks that a feasible solution exists. 1. TNTR~DUCTI~N In applications of probability theory to the physical sciences [l] one is often faced with the problem of determining a distribution consistent with a given set of average values. For n distinct states one thus seeks a vector x Here Eq. (1) is the normalization condition and (2) defines b, as the average value of the property A, , whose magnitude in the ith state is A,$. Equations (1) and (2) represent 111 + 1 constraints on the vector x and hence, if m < n-1, do not suffice to provide a unique characterization. The principle of maximal entropy [l] provides that when IIT < n-1, the probability assignment be made by the additional condition that the entropy, S[x] (or missing information [l, 21) of the distribution, S[x] =-i xi In xi, i=l